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DETAILED ACTION 



Specification 



1 . The abstract of the disclosure is objected to because it contains reference 
numerals from the drawings. Correction is required. See MPEP § 608.01(b). 

2. The disclosure is objected to because of the following informalities: 
On page 20, line 16, "classifier 614" should be -classifier 624-. 
Appropriate correction is required. 

Claim Objections 

3. Claims 10 and 15 are objected to because of the following informalities: 

In claim 10, line 3, "with the each classifier" should be -with the each classifier-. 
In claim 15, line 2, "three-dimension image" should be -three-dimensional 
image-. 

Appropriate correction is required. 

4. The following is a quotation of 37 CFR 1 .75(d)(1 ): 

The claim or claims must conform to the invention as set forth in the remainder of the specification and 
the terms and phrases used in the claims must find clear support or antecedent basis in the description 
so that the meaning of the terms in the claims may be ascertainable by reference to the description. 
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5. Claims 2 and 6 are objected to under 37 CFR 1 .75(d)(1), as failing to conform to 
the invention as set forth in the remainder of the specification. 

Claim 2 requires a feature vector to apply a classifier grid model to the input 
image. However, when the claim is read in light of the specification, "feature vector" 
should be "feature extractor", as it is stated that "A given feature extractor utilizes a 
classifier grid model" at page 20, line 18. Therefore, in line 1 of the claim, "given feature 
vector extractor" will be assumed for examination purposes. 

Claim 6 requires an image source to include a stereo camera. A stereo camera 
is a camera that contains two lenses and sensors within a single structure, which is not 
supported by the disclosure. However, when the claim is read in light of the 
specification, "stereo camera" should be "stereo camera system", as it is stated that "the 
cameras 70, 72 are charge-coupled devices ("CCD") or complementary metal-oxide 
semiconductor ("CMOS") devices" at page 7, line 18. Therefore, in line 2 of the claim, 
"a stereo camera system" will be assumed for examination purposes. 

6. The following is a quotation of 37 CFR 1 .75(a): 

The specification must conclude with a claim particularly pointing out and distinctly claiming- the subject 
matter which the applicant regards as his invention or discovery. 

7. Claims 17 and 18 are objected to under 37 CFR 1.75(a), as failing to particularly 
point out and distinctly claim the subject matter which application regards as his 
invention or discovery. 

Claim 17 lacks antecedent basis for "the classifier" at line 4. The following will be 
assumed for examination purposes: "the pattern recognition classifier". 
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Claim 18 lacks antecedent basis for "the classifier" at line 3. The following will be 
assumed for examination purposes: "the pattern recognition classifier". 

Claim Rejections - 35 USC § 103 

8. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

9. Claim 1, 2, 4-7, 10-14, 16-22 and 24-26 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over the combination of Owechko et al. (US 6,801 ,662) and Smith 
et al. ("Quad-Tree Segmentation for Texture-Based Image Query", ACM article). 

Regarding claims 1, 2, 10, and 11, Owechko discloses a system for object 
classification for use in a vehicle system ("systems and methods for detection and 
classification of objects for use in control of vehicle systems, such as air bag 
deployment systems" at col. 1, line 8) comprising 

a vision system that images a vehicle interior to provide an input image ("system 
for detecting and tracking objects within a specified area that can be adapted for 
detecting and recognizing occupants within a vehicle" at col. 2, line 27; "vision-based 
system" at col. 2, line 39) 
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a plurality of pattern recognition classifiers (figure 1, numerals 135, 145, 155), 
each pattern recognition classifier operative to process feature data associated with the 
input image to determine an associated output class of the input image ("operate on the 
feature data to classify the occupant into one of a small number of classes" at col. 4, 
line 49); and 

a plurality of feature extractors (figure 1, numerals 110, 120, 130, 140), each 
feature extractor extracting feature data from the input image for an associated one of 
the plurality of pattern recognition classifiers ("Feature extraction modules 110, 120, 
130, and 140 receive and process frames from the stream of images 105 to provide 
feature data" at col. 4, line 46). 

Owechko does not teach extracting feature data according to a classifier grid 
model representing the associated classifier wherein a given feature extractor applies 
the classifier grid model representing its associated classifier to the input image to 
produce a plurality of sub-images, the feature extractor extracting feature data relating 
to each of the plurality of sub-images and each grid model is derived from training 
images associated with the at least one associated output class. 

Smith teaches a method of image segmentation to extract feature information 
based on a classifier grid model ("Using a quad-tree approach to image segmentation, 
feature sets are extracted from image blocks" at section 7, line 4) wherein the grid 
model is derived from training images ("training will construct discriminant functions 
general enough to discriminate between new and unknown textures" at section 4.1 , line 

9). 
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It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify the feature extraction modules of Owechko using the quad-tree 
segmentation taught by Smith as described above, as an "efficient approach towards 
segmenting and representing textures... and provides a general framework by which 
other discriminating features can be used for image segmentation" (Smith at section 7, 
line 17). 

Regarding claim 4, Owechko discloses a system wherein at least on of the 
plurality of pattern recognition classifiers includes an artificial neural network ("an NDA 
network is used to generate class confidences" at col. 10, line 57). 

Regarding claims 5 and 6, Owechko discloses a system further comprising an 
image source that provides the input image wherein the image source includes a stereo 
camera system ("Means for capturing images of an area may comprise CMOS or CCD 
cameras" at col. 3, line 10; "stereo imaging system" at col. 7, line 58). 

Regarding claim 7, Owechko discloses a system further comprising an arbitrator 
associated with the plurality of pattern recognition classifiers that evaluates a plurality of 
outputs associated with the classifiers and determines an associated output class for 
the input image from the plurality of classifier outputs ("predictions and confidences of 
the classifiers are then combined in a sensor fusion engine 170 which makes the final 
decision" at col. 4, line 56). 

Regarding claims 12 and 13, Owechko discloses a system wherein at least one 
of the plurality of output classes represents a human adult seated within the vehicle and 
at least one of the plurality of output classes represents a rearward facing infant seat 
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positioned within the vehicle interior ("four classifications: adult in normal or twisted 
position, adult out-of-position (OOP), rear-facing infant seat (RFIS)" at col. 5, line 49). 

Regarding claim 14, Owechko discloses a system wherein at least one of the 
plurality of output classes represents a human head ("when an occupant is out of 
position with a vision based algorithm capable of estimating the occupant position in 
real time and then tracking dynamically his head" at col. 7, line 34). 

Regarding claims 16-22, and 24-26, Owechko discloses a method for classifying 
image data into one of a plurality of output classes comprising 

imaging an unknown object to create an input image which includes imaging an 
occupant of a passenger seat within a vehicle; 

extracting feature data from a plurality of images including extracting a set of at 
least one feature value from each image ("The o (d) *s are also stored in a vector" at col. 
6, line 60; "The edge densities of each cell in the edge density map are stacked as 
features... provided by a feature vector" at col. 9, line 28; "motion density map cell form 
the feature vector" at col. 10, line 40) wherein the feature value includes a contrast 
measure ("The a (d) 's are also stored in a vector" at col. 6, line 60) 

classifying the unknown object from the extracted feature data. 

Owechko does not teach establishing a classifier grid model associated with a 
pattern recognition classifier, overlaying the classifier grid model over the input image to 
produce a plurality of sub-images such that each sub-image provides an equal number 
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of feature values to an associated feature vector wherein the feature values include a 
coarseness measure. 

Smith teaches a method comprising 
establishing a classifier grid model including 

generating a representative image that represents at least one 
output class associated with the associated classifier ("An image database 
was generated by randomly compositing five cuts per image" at section 
5.1, line 1) including 

combining a plurality of training images associated 
with the classifier ("five cuts per image from 134 images 
consisting of 1 12 Brodatz texture images and 22 real world 
images" at section 5.1, line 2), 
dividing the representative image according to an initial grid pattern 
to form a plurality of sub-images "'Queries-by-texture' were the performed 
on this composite image database" at section 5.1 , line 12), 

identifying at least one sub-image formed by said grid pattern 
having at least one attribute of interest (figure 2, "feature extraction"), and 
modifying said grid pattern in response to the identified at least one 
sub-image having said at least one attribute of interest so as to form a 
modified pattern (figure 2, "feature extraction; figure 4a) including 
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modifying the grid pattern to divide the identified sub- 
images into respective pluralities of sub-images (figure 2, 
"feature extraction"; figure 4a) 
wherein the identifying at least one sub-image and modifying the 
grid pattern in response to the identified sub-image are repeated iteratively 
until a termination event is recorded wherein the termination event 
comprises producing a modified grid that divides the class composite 
image into a threshold number of sub-images ('When no children are 
close, all children are kept as nodes, and quad-tree iteration continues on 
each child" at section 4.2.4, line 17) 
overlaying the classifier grid model over the input image to produce a plurality of 
sub-images (figure 4a) such that each sub-image provides an equal number of feature 
values to an associated feature vector (figure 2, "feature extraction") wherein the feature 
values include a contrast measure ("texture segmentation by grouping spatial image 
blocks" at section 2.2, paragraph 2, line 1 ; "treat each block as a separate image of 
texture" at section 3.2, line 1). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify the feature extraction modules of Owechko using the quad-tree 
segmentation taught by Smith as described above, as an "efficient approach towards 
segmenting and representing textures... and provides a general framework by which 
other discriminating features can be used for image segmentation" (Smith at section 7, 
line 17). 
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10. Claims 3 and 8 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
the combination of Owechko and Smith as applied to claim 1 above, and further in view 
of Gokturk et al. (US 2004/0153229; hereinafter referred to as Gokturk '229, which 
incorporates teachings from Gokturk et al. (US 2003/0169906; hereinafter referred to as 
Gokturk '906)). 

Regarding claim 3, the Owechko and Smith combination teach the elements of 
claim 1 as shown in the 103 rejection above. 

The Owechko and Smith combination does not teach a system wherein at least 
one of the plurality of pattern recognition classifiers includes a support vector machine. 

Gokturk '229 discloses a system in the same field of endeavor of vehicle safety 
("relates to intelligent deployment and use of such airbags and safety restraints for 
vehicles" at paragraph 0002, line 3) wherein at least one of the plurality of pattern 
recognition classifiers includes a support vector machine ("images are next fed into a 
training algorithm such as nearest neighbor classification, support vector machines" in 
paragraph 0147, line 17). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify the classifier modules of Owechko and Smith using the 
algorithm taught by Gokturk '229 as described above, to find "the best differentiating 
hypersurface. . .that optimally differentiates the data belonging to the particular class C if 
from the rest of the data belonging to any other" (Gokturk '906 at paragraph 0085, line 
5). 
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Regarding claim 8, the Owechko and Smith combination teaches the elements 
of claim 1 as described in the 103 rejection above. 

The Owechko and Smith combination does not teach an image preprocessor that 
removes background information and noise from the input image. 

Gokturk '906, referenced by Gokturk '229, teaches an image preprocessor that 
removes background information ("subtracting the background image from the image" at 
paragraph 0068, line 3) and noise from the input image ("morphological opening 
operation may be executed to remove the noise" at paragraph 0068, line 13). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify the pattern recognition classifier assembly of Owechko and 
Smith using the preprocessing taught by Gokturk '229 as described above, to eliminate 
"unwanted portions of the depth image (or alternatively the intensity image)" (Gokturk 
'906 at paragraph 0068, line 10). 

11. Claim 15 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Owechko and Smith as applied to claims 1 and 1 1 above, and further in 
view of Krumm (US 5,983,147). 

The Owechko and Smith combination teaches the elements of claim 1 1 as 
described in the 103 rejection above. 

The Owechko and Smith combination does not teach producing three-dimension 
image data of the vehicle interior as a stereo disparity map. 
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Krumm teaches producing three-dimension image data of the vehicle interior as 
a stereo disparity map ("disparity image-one that gives disparity values at every point in 
the image" at col. 6, line 17). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify the vision system of Owechko and Smith using the disparity 
map taught by Krumm as described above, to provide "an invariant image for 
classification" (Krumm at col. 6, line 14) that eliminates the need to "compute range 
values" (Krumm at col. 6, lines 1 1-14). 

12. Claim 9 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Owechko, Smith and Gokturk '229 as applied to claim 8 above, and 
further in view of Ericksen et al. ("MAHEM: A Multiprocessor...", SPIE article). 

The Owechko, Smith and Gokturk combination teach the elements of claim 8 

above. 

The Owechko, Smith and Gokturk combination does not teach applying a 
contrast limited adaptive histogram equalization that adjusts the image for lighting 
conditions. 

Ericksen teaches a contrast limited adaptive histogram equalization that adjusts 
the image for lighting conditions ("Contrast Limited Adaptive Histogram Equalizaton 
(CLAHE)" in section I, line 3; "sensitivity to the intensity distribution is limited to visually 
relevant (nearby) intensities" in section III, paragraph 1, line 8). 
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It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify the image preprocessor of Owechko, Smith and Gokturk using 
the histogram equalization taught by Ericksen as described above, to not "enhance 
noise in relatively constant-level areas of the image" (Ericksen at section III, paragraph 
4, line 2). 

13. Claim 23 is rejected under 35 U.S.C. 103(a) as being unpatentable over the 
combination of Owechko and Smith as applied to claim 22 above, and further in view of 
Kaplan et al. ("Texture Segmentation using Multiscale Hurst Features", IEEE Article). 

The Owechko and Smith combination teaches the elements of claim 22 as 
described in the 103 rejection above. 

The Owechko and Smith combination does not teach a feature value including an 
average grayscale value. 

Kaplan teaches a feature value including an average grayscale value ("mean 
feature is computed as the average grayscale" at section 4.1, paragraph 3, line 1). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to modify the feature extractors of Owechko and Smith using the mean 
feature taught by Kaplan as described above, to provide another measure for 
consideration that would improve classification accuracy. 
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Conclusion 



14. The prior art made of record and not relied upon is considered pertinent to 
applicants disclosure. US 6,608,910, US 6,856,694 and US 6,507,779 are each 
pertinent as teaching vehicle safety systems. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Katrina Fujita whose telephone number is (571 ) 270- 
1574. The examiner can normally be reached on M-Th 8-5:30pm, F 8-4:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Brian P. Werner can be reached on (571 ) 272-7401 . The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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